Matching, Searching, and Replacing Text Patterns in Python
Course: Python Training
Topic: Matching, Searching, and Replacing Text Patterns
Course Link: Python Training Course at JustAcademy
Demo Link: Register for a Course Demo
Official Website: JustAcademy
1. Introduction to Text Pattern Matching
Text pattern matching is the process of identifying specific characters, words, numbers, or text structures within a string. Python provides several built-in string methods and the re module to match, search, extract, split, and replace text according to defined patterns.
The re module supports regular expressions, commonly called regex. A regular expression is a sequence of characters that describes a text pattern. Instead of checking every character manually, developers can use a compact expression to identify items such as email-like strings, phone numbers, dates, product codes, and repeated words.
For example, a program might need to find every number in a sentence, replace extra spaces, extract usernames from email addresses, or check whether a string follows a particular format.
Learning Objectives
- Understand text matching, searching, and replacement.
- Learn the difference between literal string operations and regular expressions.
- Use the Python
re module.
- Understand regex metacharacters, character classes, and quantifiers.
- Use
re.search(), re.match(), re.fullmatch(), and re.findall().
- Extract matched text using groups and match objects.
- Replace text using
re.sub() and re.subn().
- Use regex flags, compiled patterns, and lookarounds.
- Build practical text-processing applications.
2. Understanding Strings and Text Operations
A string is a sequence of characters enclosed in single quotes, double quotes, or triple quotes. Python provides built-in methods for straightforward text operations without requiring regular expressions.
Example: Searching for a Word
message = "Python is useful for automation."
print("Python" in message)
print(message.find("useful"))
print(message.startswith("Python"))
print(message.endswith("automation."))
Output:
True
10
True
True
Common String Methods
| Method or Operator |
Purpose |
Example |
in |
Checks whether text exists in a string. |
"cat" in "catalog" |
find() |
Returns the first matching position or -1. |
"hello".find("e") |
index() |
Returns the matching position or raises an error. |
"hello".index("e") |
replace() |
Replaces literal occurrences of text. |
"red car".replace("red", "blue") |
startswith() |
Checks the beginning of a string. |
"Python".startswith("Py") |
endswith() |
Checks the ending of a string. |
"file.txt".endswith(".txt") |
split() |
Splits a string into parts. |
"a,b".split(",") |
strip() |
Removes whitespace from both ends by default. |
" hello ".strip() |
Use ordinary string methods when you need to find or replace exact text. Use regular expressions when the text to match follows a pattern rather than one fixed sequence of characters.
3. Introduction to the re Module
Python's re module provides regular expression operations. It is part of the Python standard library, so it does not require a separate installation.
Importing the Module
import re
Basic Example
import re
text = "Learn Python programming"
pattern = r"Python"
result = re.search(pattern, text)
if result:
print("Pattern found:", result.group())
else:
print("Pattern not found")
Output:
Pattern found: Python
The expression r"Python" is the pattern, and text is the string being searched. The search() function returns a match object when a match is found; otherwise, it returns None.
Why Use Raw Strings?
Regular expressions often contain backslashes. Python also uses backslashes in ordinary string literals for escape sequences. Prefixing a regex pattern with r reduces confusion by preserving backslashes as written in the string literal.
import re
text = "Order number: 4582"
pattern = r"\d+"
match = re.search(pattern, text)
if match:
print(match.group())
Output:
4582
4. Understanding Regular Expression Metacharacters
Metacharacters are characters that have a special meaning in regular expressions. They let you describe flexible patterns rather than exact strings.
| Metacharacter |
Meaning |
Example |
. |
Matches any character except a newline by default. |
c.t matches cat and cut. |
^ |
Matches the start of a string or line, depending on flags. |
^Hello |
$ |
Matches the end of a string or line, depending on flags. |
end$ |
* |
Matches zero or more repetitions of the preceding item. |
ab* |
+ |
Matches one or more repetitions. |
\d+ |
? |
Matches zero or one repetition. |
colou?r |
{n} |
Matches exactly n repetitions. |
\d{4} |
{n,m} |
Matches between n and m repetitions. |
\d{2,4} |
[] |
Matches one character from a character set. |
[aeiou] |
() |
Groups expressions and can capture matched text. |
(ab)+ |
| |
Represents alternatives. |
cat|dog |
\ |
Introduces special sequences or escapes a metacharacter. |
\. matches a literal period. |
Example: Matching Different Words
import re
text = "I like cats and dogs."
pattern = r"cat|dog"
print(re.findall(pattern, text))
Output:
['cat', 'dog']
The pipe operator matches either alternative. Here, the pattern finds the substring cat within cats and dog within dogs.
5. Common Regex Character Classes and Special Sequences
Character classes let you match groups of characters without listing every possible character individually.
| Pattern |
Meaning |
Example |
\d |
A digit character. |
\d+ |
\D |
A character that is not a digit. |
\D+ |
\w |
A word character; for Unicode strings, this includes Unicode alphanumeric characters and underscore. |
\w+ |
\W |
A character that is not a word character. |
\W+ |
\s |
Whitespace such as spaces, tabs, and newlines. |
\s+ |
\S |
A non-whitespace character. |
\S+ |
[a-z] |
A lowercase ASCII letter. |
[a-z]+ |
[A-Z] |
An uppercase ASCII letter. |
[A-Z]+ |
[0-9] |
A digit from 0 through 9. |
[0-9]{3} |
[^0-9] |
A character other than an ASCII digit. |
[^0-9]+ |
Example: Extracting Words and Numbers
import re
text = "Python 3 has 100 useful features."
words = re.findall(r"\w+", text)
numbers = re.findall(r"\d+", text)
print(words)
print(numbers)
Output:
['Python', '3', 'has', '100', 'useful', 'features']
['3', '100']
findall() returns all non-overlapping matches. When the pattern includes capturing groups, the result may contain strings or tuples representing those groups.
6. Understanding Quantifiers
Quantifiers define how many times a character, character class, or grouped expression may appear.
| Quantifier |
Meaning |
Example |
* |
Zero or more repetitions. |
ab* matches a, ab, and abbb. |
+ |
One or more repetitions. |
ab+ matches ab and abbb, but not a. |
? |
Zero or one repetition. |
colou?r matches color and colour. |
{3} |
Exactly three repetitions. |
\d{3} matches three consecutive digits. |
{2,5} |
Between two and five repetitions. |
a{2,5} |
{2,} |
At least two repetitions. |
\d{2,} |
Example
import re
text = "a ab abb abbb abbbb"
pattern = r"ab+"
print(re.findall(pattern, text))
Output:
['ab', 'abb', 'abbb', 'abbbb']
The pattern ab+ requires the letter a followed by at least one b. It does not match the standalone a.
7. Greedy and Non-Greedy Matching
By default, repetition operators such as * and + are greedy: they attempt to match as much text as possible while allowing the overall pattern to succeed. Adding ? after a repetition operator makes it non-greedy, meaning it attempts to match as little as possible.
Greedy Example
import re
text = "<b>Python</b> and <b>SQL</b>"
print(re.findall(r"<.*>", text))
Output:
['<b>Python</b> and <b>SQL</b>']
Non-Greedy Example
import re
text = "<b>Python</b> and <b>SQL</b>"
print(re.findall(r"<.*?>", text))
Output:
['<b>', '</b>', '<b>', '</b>']
The non-greedy version stops at the earliest possible closing angle bracket for each match. However, regex is not a full HTML parser; for complex HTML, use a dedicated HTML parsing library.
8. Matching at the Start, Searching Anywhere, and Matching the Entire String
Python offers different functions depending on where the pattern should match.
| Function |
Purpose |
Return Value |
re.match() |
Checks for a match at the beginning of the string. |
A match object or None. |
re.search() |
Searches for the first match anywhere in the string. |
A match object or None. |
re.fullmatch() |
Checks whether the entire string matches the pattern. |
A match object or None. |
Using re.match()
import re
text = "Python programming"
result = re.match(r"Python", text)
if result:
print("Matched at the beginning")
else:
print("No match at the beginning")
Output:
Matched at the beginning
Using re.search()
import re
text = "I am learning Python programming."
result = re.search(r"Python", text)
if result:
print("Found:", result.group())
print("Start:", result.start())
print("End:", result.end())
Output:
Found: Python
Start: 13
End: 19
Using re.fullmatch()
import re
code = "ABC123"
if re.fullmatch(r"[A-Z]{3}\d{3}", code):
print("Valid code")
else:
print("Invalid code")
Output:
Valid code
Use fullmatch() when the entire input must satisfy the pattern, such as a product code or a field with a clearly defined format. It does not replace all business-rule validation.
9. Finding All Matches With findall()
The re.findall() function returns all non-overlapping matches in a string. It is useful when extracting multiple values, such as prices, identifiers, dates, or words.
Example: Extracting Prices
import re
text = "Laptop costs 55000, mouse costs 800, and keyboard costs 1500."
prices = re.findall(r"\d+", text)
print(prices)
Output:
['55000', '800', '1500']
Notice that the returned numbers are strings. Convert them to integers or floating-point values when numeric calculations are required.
price_values = [int(value) for value in prices]
print(sum(price_values))
Output:
57300
Finding Words Starting With a Capital Letter
import re
text = "Python, Java, and JavaScript are programming languages."
capitalized_words = re.findall(r"\b[A-Z][a-zA-Z]*\b", text)
print(capitalized_words)
Output:
['Python', 'Java', 'JavaScript']
10. Iterating Over Matches With finditer()
The re.finditer() function returns an iterator of match objects. Unlike findall(), it makes match positions and groups available for each occurrence.
import re
text = "Order 101 costs 500. Order 102 costs 750."
for match in re.finditer(r"\d+", text):
print(
"Value:", match.group(),
"Position:", match.span()
)
Output:
Value: 101 Position: (6, 9)
Value: 500 Position: (17, 20)
Value: 102 Position: (27, 30)
Value: 750 Position: (38, 41)
Useful Match Object Methods
| Method |
Purpose |
group() |
Returns the full matched text. |
group(1) |
Returns the text captured by the first capturing group. |
groups() |
Returns the captured groups as a tuple. |
start() |
Returns the starting index of the match. |
end() |
Returns the ending index of the match. |
span() |
Returns the start and end indexes as a tuple. |
11. Capturing Groups and Group Extraction
Parentheses create capturing groups in a regular expression. Groups allow you to extract individual parts of a matched pattern instead of returning only the complete match.
Example: Extracting a Date
import re
text = "Meeting date: 22-10-2026"
pattern = r"(\d{2})-(\d{2})-(\d{4})"
match = re.search(pattern, text)
if match:
print("Full date:", match.group(0))
print("Day:", match.group(1))
print("Month:", match.group(2))
print("Year:", match.group(3))
Output:
Full date: 22-10-2026
Day: 22
Month: 10
Year: 2026
Named Capturing Groups
Named groups give meaningful names to captured portions of a pattern. They improve readability in more complicated expressions.
import re
text = "Employee ID: EMP-2048"
pattern = r"(?P[A-Z]+)-(?P\d+)"
match = re.search(pattern, text)
if match:
print(match.group("prefix"))
print(match.group("number"))
print(match.groupdict())
Output:
EMP
2048
{'prefix': 'EMP', 'number': '2048'}
Named groups are particularly useful for extracting structured data from logs, records, and consistently formatted text.
12. Non-Capturing Groups
A non-capturing group uses the syntax (?:...). It groups part of a pattern without including that group in the captured results.
import re
text = "color colour"
pattern = r"colou?r"
print(re.findall(pattern, text))
Output:
['color', 'colour']
In this example, the optional u is already handled by the quantifier. Non-capturing groups are useful when grouping a larger expression without adding unnecessary captured values.
import re
text = "cat dog bird"
pattern = r"(?:cat|dog)"
print(re.findall(pattern, text))
Output:
['cat', 'dog']
13. Replacing Text With re.sub()
The re.sub() function searches for non-overlapping matches and replaces them with replacement text. It returns a new string; the original string remains unchanged.
Basic Replacement
import re
text = "I like Java. Java is popular."
updated_text = re.sub(r"Java", "Python", text)
print(updated_text)
Output:
I like Python. Python is popular.
Replacing All Digits
import re
text = "My order number is 4582."
updated_text = re.sub(r"\d", "*", text)
print(updated_text)
Output:
My order number is ****.
Limiting the Number of Replacements
The optional count argument limits how many matches are replaced. Its default value of zero means all non-overlapping matches are replaced.
import re
text = "red red red red"
print(re.sub(r"red", "blue", text, count=2))
Output:
blue blue red red
14. Replacing Text Using Captured Groups
Captured groups can be reused in replacement strings. This is helpful when changing a date format, rearranging a name, or restructuring an identifier.
Example: Rearranging a Date
import re
date = "22-10-2026"
updated_date = re.sub(
r"(\d{2})-(\d{2})-(\d{4})",
r"\3/\2/\1",
date
)
print(updated_date)
Output:
2026/10/22
In replacement strings, \1, \2, and \3 refer to captured groups. The clearer form \g<1> or \g<name> can avoid ambiguity when text follows a group reference.
Example With Named Groups
import re
text = "Asha Sharma"
pattern = r"(?P\w+)\s+(?P\w+)"
updated = re.sub(
pattern,
r"\g, \g",
text
)
print(updated)
Output:
Sharma, Asha
15. Replacing Text With a Function
The replacement argument of re.sub() can be a function. Python passes each match object to the function, and the returned string becomes the replacement text.
Example: Doubling Numbers
import re
def double_number(match: re.Match[str]) -> str:
number = int(match.group())
return str(number * 2)
text = "Values: 10, 20, 35"
updated_text = re.sub(r"\d+", double_number, text)
print(updated_text)
Output:
Values: 20, 40, 70
When Function-Based Replacement Is Useful
- Transforming numbers or codes.
- Applying conditional replacement rules.
- Formatting matched values dynamically.
- Performing more complex replacements than a fixed string allows.
16. Counting Replacements With re.subn()
The re.subn() function works similarly to re.sub(), but returns a tuple containing the updated string and the number of replacements performed.
import re
text = "cat dog cat bird cat"
updated_text, count = re.subn(r"cat", "fox", text)
print(updated_text)
print("Replacements:", count)
Output:
fox dog fox bird fox
Replacements: 3
This is useful when you need to confirm how many occurrences were changed during text cleanup or data transformation.
17. Splitting Text With re.split()
The re.split() function splits a string wherever the pattern matches. It is useful when separators vary, such as multiple spaces, commas, semicolons, or hyphens.
import re
text = "apple,banana;orange grape"
items = re.split(r"[,;\s]+", text)
print(items)
Output:
['apple', 'banana', 'orange', 'grape']
Limiting Splits
import re
text = "one,two,three,four"
print(re.split(r",", text, maxsplit=2))
Output:
['one', 'two', 'three,four']
Use the maxsplit argument to limit the number of splits. Capturing groups in the separator pattern may also cause separator text to appear in the result.
18. Regex Flags
Flags modify how a regular expression behaves. They can be passed as the third argument to functions such as re.search() or included when compiling a pattern.
| Flag |
Purpose |
Example |
re.IGNORECASE or re.I |
Matches letters without case sensitivity. |
python can match Python. |
re.MULTILINE or re.M |
Makes ^ and $ work at line boundaries as well as string boundaries. |
Find patterns at the beginning of lines. |
re.DOTALL or re.S |
Allows the dot . to match newline characters. |
Match across multiple lines. |
re.VERBOSE or re.X |
Allows whitespace and comments in many parts of a pattern for readability. |
Write complex patterns across several lines. |
re.ASCII or re.A |
Makes certain character classes use ASCII-only behavior for Unicode string patterns. |
Restrict \w and related classes to ASCII behavior. |
Case-Insensitive Search
import re
text = "Python PYTHON python"
matches = re.findall(r"python", text, re.IGNORECASE)
print(matches)
Output:
['Python', 'PYTHON', 'python']
Multiline Search
import re
text = """First line
ERROR: File missing
Second line
ERROR: Access denied"""
errors = re.findall(r"^ERROR:.*", text, re.MULTILINE)
print(errors)
Output:
['ERROR: File missing', 'ERROR: Access denied']
19. Compiling Regular Expressions
The re.compile() function creates a reusable regular expression object. Compiling a pattern is helpful when the same pattern is used repeatedly or when you want to keep a complex expression organized.
import re
pattern = re.compile(r"\b[A-Z]{2}\d{4}\b")
codes = [
"AB1234",
"CD5678",
"A1234",
"XY9999"
]
for code in codes:
if pattern.fullmatch(code):
print("Valid:", code)
else:
print("Invalid:", code)
Output:
Valid: AB1234
Valid: CD5678
Invalid: A1234
Valid: XY9999
Compiled patterns provide methods such as search(), fullmatch(), findall(), finditer(), and sub(). Python also caches some recent regular expressions internally, so explicit compilation is most useful for clarity and repeated use rather than as a guaranteed speed improvement in every case.
20. Word Boundaries and Anchors
Anchors and boundaries help control where a pattern can match. They are particularly useful when searching for complete words or validating formats.
| Expression |
Meaning |
\b |
Matches a word boundary. |
\B |
Matches a position that is not a word boundary. |
^ |
Matches the beginning of the string, or each line with multiline mode. |
$ |
Matches the end of the string or a line, depending on mode. |
\A |
Matches the beginning of the string. |
\Z |
Matches the end of the string. |
Matching a Whole Word
import re
text = "cat catalog scatter cat"
print(re.findall(r"\bcat\b", text))
Output:
['cat', 'cat']
The word boundary prevents the pattern from matching cat as part of catalog or scatter.
21. Lookahead and Lookbehind Assertions
Lookarounds allow a pattern to check surrounding text without including that surrounding text in the matched result.
Positive Lookahead
The expression (?=...) checks that a specified pattern follows the current position.
import re
text = "item10 item20 item30 itemA"
matches = re.findall(r"item(?=\d)", text)
print(matches)
Output:
['item', 'item', 'item']
Only the word item followed by a digit is selected. The digit itself is not included in the returned match.
Negative Lookahead
import re
text = "file.txt file.csv file.pdf"
matches = re.findall(r"file\.(?!csv)\w+", text)
print(matches)
Output:
['file.txt', 'file.pdf']
Positive Lookbehind
The expression (?<=...) checks that a specified pattern appears immediately before the current position. Python's standard re module requires lookbehind patterns to have a fixed length.
import re
text = "USD 500, INR 900, USD 750"
matches = re.findall(r"(?<=USD )\d+", text)
print(matches)
Output:
['500', '750']
Negative Lookbehind
import re
text = "USD500 500 USD750"
matches = re.findall(r"(?<!USD)\d+", text)
print(matches)
Output:
['500', '500', '750']
This pattern excludes digit sequences immediately preceded by USD. It does not enforce a general currency format or inspect the character before the currency text.
22. Escaping Special Characters
Some characters have special meanings in regex patterns. To match them literally, escape them with a backslash. Alternatively, use re.escape() when building a pattern from literal text supplied at runtime.
Matching a Literal Dot
import re
text = "example.com exampleXcom"
print(re.findall(r"example\.com", text))
Output:
['example.com']
The escaped dot matches an actual period instead of the regex metacharacter that means any non-newline character.
Using re.escape()
import re
search_text = "C++"
text = "I use C++ and Python."
pattern = re.escape(search_text)
print(re.findall(pattern, text))
Output:
['C++']
re.escape() is useful when you want to search for a literal value that may contain characters such as +, ., *, or ?.
23. Common Practical Applications
Application 1: Extracting Email-Like Strings
import re
text = """
Contact: [email protected]
Sales: [email protected]
Invalid entry: user@
"""
pattern = r"\b[\w.+-]+@[\w-]+(?:\.[\w-]+)+\b"
emails = re.findall(pattern, text)
print(emails)
Output:
['[email protected]', '[email protected]']
This is a useful introductory pattern, not a complete implementation of every valid email address rule. For production email workflows, combine appropriate validation with the application's verification process.
Application 2: Extracting Phone-Like Digit Sequences
import re
text = "Call 9876543210 or 9123456780."
pattern = r"\b[6-9]\d{9}\b"
numbers = re.findall(pattern, text)
print(numbers)
Output:
['9876543210', '9123456780']
This example represents a simplified 10-digit Indian mobile-number pattern. Real phone-number validation may require country codes, formatting rules, and a dedicated phone-number library.
Application 3: Removing Extra Spaces
import re
text = "Python is easy to learn."
cleaned = re.sub(r"\s+", " ", text).strip()
print(cleaned)
Output:
Python is easy to learn.
Application 4: Extracting Hashtags
import re
text = "Learning #Python and #DataScience is useful."
hashtags = re.findall(r"#[A-Za-z0-9_]+", text)
print(hashtags)
Output:
['#Python', '#DataScience']
This simplified expression handles ASCII letters, digits, and underscores. Hashtag rules can vary by platform and may need broader Unicode support.
24. Cleaning and Normalizing Text
Text normalization transforms inconsistent text into a more consistent format. It is commonly used before searching, comparing records, analyzing text, or importing data.
Example: Cleaning a Sentence
import re
text = " Python!!! is GREAT... "
text = text.strip()
text = re.sub(r"\s+", " ", text)
text = re.sub(r"[!?.]+", ".", text)
print(text)
Output:
Python. is GREAT.
The example removes leading and trailing whitespace, collapses internal whitespace, and replaces repeated punctuation with a period. It is intentionally simple; production text normalization should account for the text's purpose and language.
Replacing Multiple Different Terms
import re
text = "I use JS and TS for web development."
pattern = r"\b(JS|TS)\b"
replacements = {
"JS": "JavaScript",
"TS": "TypeScript"
}
def expand_term(match: re.Match[str]) -> str:
return replacements[match.group()]
updated = re.sub(pattern, expand_term, text)
print(updated)
Output:
I use JavaScript and TypeScript for web development.
Using a dictionary and a replacement function keeps the transformation rules explicit and easier to maintain.
25. Matching and Replacing Text in Files
Regular expressions can process the contents of text files. When working with files, specify an appropriate encoding and handle errors such as missing files or permission restrictions.
Example: Reading a Log File
import re
from pathlib import Path
file_path = Path("application.log")
try:
content = file_path.read_text(encoding="utf-8")
except OSError as error:
print(f"Unable to read file: {error}")
else:
errors = re.findall(r"(?m)^ERROR:.*$", content)
print(errors)
This example extracts lines that begin with ERROR:. The (?m) inline flag enables multiline behavior for the anchors.
Example: Saving Cleaned Text
import re
from pathlib import Path
source = Path("raw_notes.txt")
destination = Path("cleaned_notes.txt")
try:
content = source.read_text(encoding="utf-8")
cleaned = re.sub(r"[ \t]+", " ", content)
destination.write_text(cleaned, encoding="utf-8")
print("Cleaned file saved.")
except OSError as error:
print(f"File operation failed: {error}")
For important files, consider writing to a temporary file first and replacing the destination only after the output has been written successfully. Avoid overwriting original data until the transformation has been verified.
26. Common Errors and Troubleshooting
| Problem |
Cause |
Solution |
| No match is found. |
The pattern does not fit the input or is case-sensitive. |
Inspect the input and consider re.IGNORECASE when appropriate. |
| Too much text is matched. |
A greedy quantifier consumes more text than intended. |
Use a non-greedy quantifier or a more specific pattern. |
| Regex syntax error. |
Parentheses, brackets, braces, or escapes may be incorrect. |
Check the pattern carefully and test a smaller expression. |
| Special characters match unexpectedly. |
Metacharacters are interpreted as regex syntax. |
Escape literal characters or use re.escape(). |
| Only the first match is returned. |
search() finds the first occurrence. |
Use findall() or finditer() for multiple matches. |
| Incorrect replacement output. |
Capture-group references may be incorrect. |
Check group numbering or use named groups. |
| Newlines are not matched by a dot. |
The dot does not match newlines by default. |
Use re.DOTALL if appropriate. |
| Validation accepts unwanted input. |
The pattern matches only a substring. |
Use fullmatch() when the entire string must match. |
| Pattern is slow on large input. |
A complex pattern may cause excessive backtracking. |
Simplify the pattern, limit input size, and avoid ambiguous nested quantifiers. |
Debugging Example
import re
text = "Order ID: AB1234"
pattern = r"[A-Z]{2}\d{4}"
match = re.search(pattern, text)
if match is None:
print("No match found")
else:
print("Matched text:", match.group())
print("Position:", match.span())
Output:
Matched text: AB1234
Position: (10, 16)
27. Best Practices for Regular Expressions
- Use raw strings such as
r"\d+" for regex patterns.
- Prefer ordinary string methods for simple literal searches and replacements.
- Use
search() for finding a pattern anywhere in a string.
- Use
fullmatch() when the entire input must satisfy a pattern.
- Use
findall() for a list of matches and finditer() when match positions are needed.
- Use named groups for complex patterns that extract multiple fields.
- Choose greedy or non-greedy quantifiers deliberately.
- Escape literal metacharacters or use
re.escape() for dynamic literal searches.
- Use
re.compile() when a reusable pattern improves clarity.
- Validate extracted data separately when business rules require more than pattern matching.
- Test empty strings, unexpected characters, Unicode input, and boundary cases.
- Avoid unnecessarily complex expressions that are difficult to maintain.
- Be cautious with user-supplied patterns and pathological backtracking on large input.
- Do not rely on a simple regex to parse complex HTML, XML, or every possible email or phone-number format.
28. Practical Project: Text Pattern Analyzer and Cleaner
This project reads a paragraph, extracts useful information, identifies email-like strings, finds numbers, and cleans extra whitespace. It combines several important operations from the re module.
Project Requirements
- Accept a text paragraph.
- Extract words and numbers.
- Find email-like strings.
- Replace repeated whitespace with a single space.
- Display the cleaned text and basic counts.
Complete Python Code
import re
def analyze_text(text: str) -> dict[str, object]:
cleaned_text = re.sub(r"\s+", " ", text).strip()
words = re.findall(r"\b\w+\b", cleaned_text)
numbers = re.findall(r"\d+", cleaned_text)
email_pattern = r"\b[\w.+-]+@[\w-]+(?:\.[\w-]+)+\b"
emails = re.findall(email_pattern, cleaned_text)
return {
"cleaned_text": cleaned_text,
"word_count": len(words),
"numbers": numbers,
"emails": emails
}
sample_text = """
Python automation is useful.
Contact [email protected].
There are 25 tasks and 3 projects.
"""
result = analyze_text(sample_text)
print("Cleaned text:", result["cleaned_text"])
print("Word count:", result["word_count"])
print("Numbers:", result["numbers"])
print("Emails:", result["emails"])
Expected Output
Cleaned text: Python automation is useful. Contact [email protected]. There are 25 tasks and 3 projects.
Word count: 13
Numbers: ['25', '3']
Emails: ['[email protected]']
Project Explanation
re.sub() normalizes whitespace.
re.findall() extracts words, numbers, and email-like strings.
len() calculates the number of extracted words.
- The returned dictionary organizes the results for later use.
Extension tasks: Add URL extraction, count how many times each word appears, allow case-insensitive search, and save the results to a JSON file.
29. Practice Exercises
Beginner Exercises
- Write a regex to find every digit in a string.
- Extract all words from a sentence using
re.findall().
- Search for the word
Python in a paragraph.
- Replace every occurrence of
Java with Python.
- Remove repeated spaces from a sentence.
Intermediate Exercises
- Extract all three-digit numbers from a string.
- Find every word that starts with an uppercase ASCII letter.
- Extract dates formatted as
DD-MM-YYYY.
- Replace a date in
DD-MM-YYYY format with YYYY/MM/DD.
- Find email-like strings in a block of text.
- Use named groups to extract a product prefix and numeric identifier.
Advanced Exercises
- Use a compiled regex to validate a product code such as
AB1234.
- Use positive lookahead to find words followed by a number.
- Use
re.sub() with a callback to double every number in a sentence.
- Extract all error lines from a multiline log file.
- Create a text-cleaning function that normalizes whitespace and selected punctuation.
30. Interview Questions and Answers
Q1. What is a regular expression?
Answer: A regular expression is a pattern used to search, match, extract, split, or replace text based on character sequences and rules.
Q2. What is the purpose of Python's re module?
Answer: The re module provides regular expression functionality in Python, including searching, matching, substitution, splitting, and extracting matched groups.
Q3. What is the difference between re.match() and re.search()?
Answer: re.match() checks for a match at the beginning of the string, while re.search() looks for the first match anywhere in the string.
Q4. What does re.fullmatch() do?
Answer: It checks whether the entire string matches the regular expression.
Q5. What is the difference between findall() and finditer()?
Answer: findall() returns matched values in a list. finditer() returns an iterator of match objects that provide match text, groups, and positions.
Q6. What is the purpose of re.sub()?
Answer: It replaces every non-overlapping match of a pattern by default and returns the updated string.
Q7. What is the difference between re.sub() and re.subn()?
Answer: re.sub() returns the updated string, while re.subn() returns the updated string and the number of replacements.
Q8. Why should raw strings be used for regex patterns?
Answer: Raw strings reduce conflicts between Python string escape sequences and regular expression backslashes, making patterns easier to write and read.
Q9. What is a capturing group?
Answer: A capturing group uses parentheses to capture part of a match so that it can be retrieved or reused in a replacement.
Q10. What is a non-capturing group?
Answer: A non-capturing group uses (?:...) to group a pattern without storing that group's text as a capture.
Q11. What is the difference between greedy and non-greedy matching?
Answer: Greedy quantifiers attempt to match as much text as possible, while non-greedy quantifiers attempt to match as little as possible while allowing the overall pattern to succeed.
Q12. What is re.compile() used for?
Answer: It creates a reusable regex pattern object that can be used with methods such as search(), findall(), and sub().
Q13. What does re.IGNORECASE do?
Answer: It makes pattern matching case-insensitive according to Python's regex rules.
Q14. What is a lookahead assertion?
Answer: A lookahead checks whether a pattern appears after the current position without including that text in the match.
Q15. Why can regex validation be insufficient?
Answer: A pattern may check only a simplified format and cannot necessarily verify whether a value exists, is genuine, or follows every real-world business rule. Additional validation may be necessary.
31. Quick Revision Checklist
- Import regular expression tools using
import re.
- Write most regex patterns as raw strings.
- Use
\d for digits, \w for word characters, and \s for whitespace.
- Use quantifiers such as
*, +, ?, and {n}.
- Use
search() to find a pattern anywhere in text.
- Use
match() for a beginning-of-string match in code compatible with current and older Python versions.
- Use
fullmatch() when the whole input must match.
- Use
findall() to collect matches and finditer() to inspect match objects.
- Use capturing groups to extract parts of a pattern.
- Use
sub() to replace text and subn() to count replacements.
- Use
split() to divide text using a pattern.
- Use regex flags to adjust case sensitivity and multiline behavior.
- Use
re.escape() when searching for dynamic literal text.
- Test patterns against valid, invalid, and edge-case inputs.
- Validate important business rules separately from regex matching.
32. Conclusion
Matching, searching, and replacing text patterns are essential Python skills for automation, data cleaning, log analysis, text extraction, and input processing. Built-in string methods are often the simplest solution for exact text operations, while the re module is useful for flexible pattern-based tasks.
By understanding character classes, quantifiers, groups, flags, lookarounds, and replacement functions, you can build text-processing solutions that are more accurate and maintainable. Start with simple patterns, test them against realistic examples, and use explicit validation whenever the result affects important application behavior.
Additional Learning Resources